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SYSTEM AND METHOD FOR EXTRACTING A FACE FROM A CAMERA PICTURE 
FOR REPRESENTATION IN AN ELECTRONIC SYSTEM 

The present invention relates graierally to computer vision technology for . 
5 recognizing images, and more parti(mlarly, to computer vision technology flmtextracte 
imagp, such as a person's fiwe, fiom a picture, for representation in an electronic qrstem. 

in present and future electronic systems, people are/will be able to meet 
each other online, in a so-caUed "virtual space" and, in order to identify themselves in this 
virtual space, fliey need to be represented by a personal icon. For example, a virtual space 
10 may be realized by any P2P^fstem(e.g., global networking). 

In anottier example, at tte entrance of an ofBce building a display could 
^ow a map of all rooms including pictures, i.e., iconic representations, of the persons who 
are currently in each room. The question arises how fliese kind of 'person icmis' may be 
easily captured, and kept up-to-date. 
15 Face reception systems may be employed for determining faces in an 

image. A representative fece recognition ^tem is described in U.S. Patent No. 6,108,437 
which B directed to a feature extraction system that processes an image for determining 
whether a fece is present in an mage and for determining its position in the hnage. One 
application of a fece recognition system is described.in U.S. Patent No. 5,781,650 which is 
20 directed to a system for determine facial features fiom an extracted image in order to 
classify a person's age ftom his/her image. U.S. Patent No. 5,987,1 54 entitted "Method 
and Means for Detecting People in Image Sequences" is directed to a system for locating a 
person in a sequence of video images to facilitate visual communication. International 
Patent Publication WO-2003 10728 is another application of a face recognition system that 
25 is targeted at privacy safeguarding. Its aim is to hide identities of people/objects by first 
detecting Ae presence of a person's personal identifying characteristics present in an 
image, and then automatically removing or replacing fliose characteristics. 

Most fece recognition systems use computer vision for recognizing identity 
of people and objects and these systems do not output (parts of) the imagp itself. The 
3 0 reference WO-2003 10728 is the only exception to fliis, but here the goal is to hide Ae 

identify of a person. 

The prior art computer vision systems are thus deficient in that none of 
these systems is used to capture and process an image of a person in onier to use it to • 
represent the person in another system, e.g., a PCC or, like system requiring mterfecing of 
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humans and machines. Hius, it would be highty desirable to provide a coii5>uter vision 
system Aat aims to present an image of a fece as representation of a person's identic in an 

organization or system. 

It is an object of flie present invention to provide a computer vision system 
and methodology that presents an image of a fiice as representation of a person's identity in 
an orgsmization or system. In accordance witti this object, the computer vision system and 
method addresses the timing/selection of a desired image. For instance, an image may be 
selected in whi«5h the subject person is gazing at the camera, or in which die person is 
smiling, eto. 

Accordingly, a computer vision system and methodology is provided that 
implements includes face recognition for capturing a picture of the person that needs to be 
represented in anoflier electronic system fliat uses or presents images, for example, and to 
extract or *cut out' flie fece fiom flat picture. The system and method is further able to 
take the snapshot on the moment that the person is gazing straight towards the camera or 
15 even at the moment that a person's fecial appearance is in a desired state, e.g.. is smiling, 
and render the extracted image suitable for representation m another electronic system. 
Thus, the invaitive system addresses the timing/selection of a desired unage to be 
extracted. 

Also provided are a computer program product for carrying out the methods 
20 of the present invention and a program storage device for the storage of the computer 

program product therein. The computer program product can be embodied in a computer- 
readable medium or contained in the multi-media stream. 

These and other features, aspects, and advantages of Ihe apparatus and 
methods of the present invention will become better understood with regard to the 
25 followfaig description, appended claims, and accompanying drawings where: 

Figure 1 illustrates a schematic view of an embodhnent of an apparatus for 
detectnig and extracting an image from a video camera output (digital stream); 

Figure 2 illustrates one application for using an extracted portrait of a 
person for identifying that person as an icon in a personal content container device 

3 0 connectable via a WiFi netwodc 

The most natural way of representing a person is by a picture of Ihe person's 
fece. However, it is hard for a person to position himself m front of a camera in such a 
way tiiat the picture has the appropriate properties of a portrait photo. Referring now to 
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Figure 1, there is illustrated a schematic of an embodiinentof an apparatus 10 for detecting 

a person in an image. 

According to the invention, a first step is processing of an imag^ to check 
for the presence of a face (using fecial features). For example, as shown m Figure 1, a 
5 camera device 15 adapted to generate stffl image or fiiU-motion video images generates, 
respectively, a first real-world imagp 12 or series of images (an image sequence), e.g., of a 
person or group of people, for example. This image may be digitized and converted to a 
digital stream 18 for mput into a image recognition processor or like device 20, for 
example, which may apply fecial recognition to capture a snapshot 22 of the target 
10 person's face fiom the first (larger) image before extracting it As known to skilled 

artisans in fece recognition and computer vision generally, the digital stream of flie image 
or image sequence may be parsed for detecting m the datastream various sub-strings which 
may comply with certam substring characteristics, viz., the characteristics of a hnman face. 
Then, computer vision techniques are utilized to "cut ouf the fece from a possible larger 
15 picture. It is understood fliat the camera device 15 and image processor 20 may be separate 
devices or embodied as a smgle stand-alone unit 

In one embodhnent, a time of the taking and receipt of Hob still image/video 
image may be programmed, and determinable by computer vision, such as at the moment 
when a person is srnilmg or when gazing towards the camera device. That is. the image 
;S0 sequence may extractihe snapshot of the target person's fece 22a when the face in the 
image is smiling or folly feces the camera. Then, the extracted image 22a is stored in a 
memory storage system or database 25. comprising non-volatile (or volatile) memory 
storage media. The extracted image may be stored together with contextual data/meta data 
such as time, place, camera, and if known that person's "personID". Finally, according to 
25 flie invention, the stored image 22a and associated contextual/metadata may be 

subsequently (or immediately) imported to or retrieved from the same system or second 
distmct system 30 executing any application requiring visual identification of the person. 
In one example, for instance, as depicted in Figure 2. the stored image 22a may be 
displayed as an icon 22b on a screen display 45 of an electronic system 30. e.g.. a mobile 
30 telephone 50. Otherwise, the extracted image may be printed on paper m a later processing 
stage. 

According to the invention, the above-described principle may be applied in 
various camera/computer vision systems: 
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1) a smart security camera fliat is adapted to create a logging list of *poitrait 
photos' of people entering or leaving a particular buildin© 

2) a portable "infotainmenf (content used for infcnmation or entratainment) 
players such as flie Portable Content Container (PCQ may support use of user icons in an 

ad-hoc sharing situatioi^ 

3) a PC or TV camera could be adapted to use Ae extracted image to create 
user icons that could be used to represent a particular person; e.g., representation in virtual 
community spaces (with advantageously, the feature Aat&e extracted photo is up-to-date); 

4) a PC, TV or bafliroom minor display system may be adapted to include 
an integrated camera that makes a daily snapshot of the person's fece. This allows the 
person to playback a sequence of images of hisAier fece, to see how his/her fece has 

changed over time; 

5) a camera-equipped mobile phone may be used to quioWy make a 
snapshot and to use this picture as a representation of the person's telephone number (caller 
ID, address book etc). The picture would pop up if this person calls Ae user's phone, for 
instance; and 

€) a camera at the entrance of a building or a reception may make snapshots 
of people .altering so that these could be used to instantly create a visitor's badge using a 
camera and computer vision at the entrance/reception of a company building. For 
exnnple, a 'fece scanning system' may be employed to continuously scan feces of people 
entering to see if they are new, or make a snapshot on explicit request. This is especially 
applicable for an organization requiring user identification badges. That is, this system is 
useful if badges with photos can be printed instantiy and especially when fomre display 
technologies become cheap and enable a visitors' badge to be electronically updated with a 
25 recent picture of the visitor or employee. This means that an employee badge could 

(theoretically) be updated every day. 

An additional feature may include the employment of flie system 
programmed to frequently capture portrait pictures of a person so fliat a fece recognition 
system may be kept up-to-date and to graduaUy adapt Ae recognition pattern to be able to 
recognize a person over a long time. For instance, the invention may feciUtate and render 
possible the maintenance of an electronic photo frame of a target user that may be 
frequently updated (e.g., daily) witii a recent picture of tiie target user. 
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Those skilled in Ihe art will appreciate fliat the present invention may be 
usefully employed in other systems where a facial image may obtained firan a first image 
and subsequently output in a useful fonn for representation in another system as a separate 
smaller image. 

5 While there has been shown and described what is considered to be 

illustrative embodiments of the invention, it will, of course, be understood Aat various 
modifications and changes in form or detail could readily be made without departing from 
the spirit of tfie invention. It is ttierefore intended that the invention be not limited to the 
exact forms described and ilhistiated, but should be constructed to cover all modifications 

10 that may fidl within tiie scope of fbs appended claims. 
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CLAIMS: 

1 . A system (10) for representing a iisw in an electronic system (30) 

comprising: 

means (15) for obtaining an imagp (12) including an image of a target 

person (22); 

means (18,20) for procesang said obtained image, said means for 
recognizing a fecial feature associated with said target person (22) in said image and 
extracting that target person's fecial portion (22A) ftom said image; 

means (45) for displaying said extracted targpt person's fecial portion in an 
electronic system display device to represent that targpt person's connection to said 

electronic system (30). 

2. The system of claim 1, wherein the means for obtaining an image 
includes an imagp capture device such as a camera (1 5) adapted to generate a still image or 

full motion image sequence. 

3. The system of claim 1, wheiem the means for processing mdudes an 
image recognition device for receiving a digital stream (18) embodying said hnage and 
applymg fecial recognition technique to extract a target person's fecial portion image. 

4. The system of clahn 2, wherem a time for taking an image ftom said 
capture device is determinable at a moment when a target person (22) is smiling or when a 
target person is gasong towards a image capture device. 

5. The system of claim 3, wherein said displaying means utilizes said 
extracted target person's facial portion (22a) as an icon (22b) for representing said target 
person via a display mterfece (45) associated with said electronic system (30). 

6. The system of claim 5, wherein said icon (22b) including said 
extracted target person's fecial portion (22a) is immediately or subsequently imported to 

said electronic system (30). 

7. The system of claim 2, further comprising memory storage device 
(25) for storing said extracted target person's fecial portion image (22a). in addition to 
cont^ctual data associated with said image. 

8. The system of claim 6, wherein said contextual data includes meto 
data including time, place, and image capture device implemented. 
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9. A method for representing a usrar in an electranic system (30) 

comprising the steps of: 

a) obtaining an image (12) mcluding an image of a target person (22); 

b) processing (20) said obtained image including recognizing a fecial 
feature associated with said target person in said image and extracting that target person's 
fecial portion (22a) from said image; and, 

c) displaying (45) said extracted target person's fecial portion in an 
electronic system (30) display device to represent that target person's connection to said 
electronic system. 

10. The method of claim 9, wherem the step a) of obtaining an image 
includes implementing an image capture device (15) such as a camera adapted to generate 
a still image or full motion image sequence. 

11. The method of claim 9. wherein the processing step b) includes the 
step of implementing an hnage recognition device (20) for receiving a digital stream (18) 
embodying said image and applying facial recognition technique to extract an hnage 
corresponding to flie target person's fecial portion (22a). 

12. The method of claim 10, wherein said step a) of obtaining (15) an 
image mcludes the step of pn>gramming a tune for taking an image from said capture 
device atamomentwhenatarget person (22) is smiling or whenatarget person is gazing 

towards a image capture device. 

13. The method of claim 1 1, wherein said displaying step includes 
utilizmg said extracted target person's fecial portion (22a) as an icon (22b) for representing 
said target person via a display mterfece (45) associated with said electronic system (30). 

14. The method of daun 13, ferther including the step of immediately or 
subsequently importing said icon (22b) includmg said extracted target person's fecial 
portion (22a) to said electronic system (30). 

15. The method of claim 10, ferther comprising the step of storing said 
extracted target person's facial portion image, in addition to contextual data associated 
with said image, in a memory storage device (25). 

16. The method of claim 14. wherein said contextual data mcludes meta 
data including time, place, and image capture device implemented. 

17. A computer program product for representing a user m an electronic 

system (30) comprising fee steps of: 
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a) obtaining an image (12) including an image of a target person ^2); 

b) processing pO) said obtained image including recognizing a fecial 
feature associated wifli said target person in said image and extracting that target person's 
fecial portion (22a) fix>m said image; and, 

c) displaying (45) said extracted target person's fecial portion in an 
electronic system (30) display device to represent that target person's connection to said 
electronic system. 

18. The computer program product of claim 17, wherein the step a) of 
obtaining an image includes implementing an image capture device (1 5) such as a camera 
adapted to generate a still image or full motion image sequence. 

19. The computer program product of claim 17, wherein the processing 
step b) includes the step of hnplementing an image recognition device (20) for receiving a 
digital stream (1 8) embodying said image and applying facial recognition technique to 
extract an image corresponding to the target person's fecial portion (22), 

20. The computer program product of claim 1 8, wherein said step a) of 
obtaining (1 5) an image includes the step of programming a time for taking an image ftom 
said image capture device at a moment when a target person (22) is smiling or when a 
target person is gazing towards a image capture device. 

21. The computer program product of claim 19, wherein said displaying 
step includes utilizing said extracted target person's facial portion (22a) as an icon (22b) 
for representing said target person via a display mterface (45) associated with said 

electronic system (30). 

22. The computer program product of claim 21 , further including the 
step of immediately or subsequently importing said icon (22b) including said extracted 
target person's facial portion (22a) to said electronic system (30). 

23 . The computer program product of claim 1 8, fiir£her comprising the 
step of storing said extracted target person's fecial portion image, in addition to contextual 
data associated with said image, in a memory storage device (25)- 

24. The computer program product of claim 22, wherein said contextual 
data includes meta data including time, place, and unage capture device implemented. 
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ABSTRACT 

A system and method for iq^nesentmg a peison in an electronic system. An imagp 
is obtained including an image of a taigetpeison. The imagp or a digital representation 
thereof, is processed to recognize a fecial feature associated wifli the target person in the 
5 image. The target person's fecial portion is flien extracted from the image. Tlie extracted 
target person's fecial portion is displayed via a display interfece associated with a same or 
separate and distinct electronic system to represent the target person when connected to 
that electronic system. An image capture device capturing the image obtamed is adapted to 
take an image at a moment when a target person is smiling or when a target person is 
10 gazmg towards capture device. An icon may be generated that includes the extracted targpt 
person's fecial portion for representing flie target user via a display interfece associated 
with the electronic system. 
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Figure 1 



